Redes Neuronales Convolucionales & Redes Recurrentes
Regresión lineal múltiple, regresión logística y Redes Neuronales
PhD. Pablo Eduardo Caicedo Rodríguez
2025-11-25
The Convolution Operation
In the context of CNNs, the operation is technically a cross-correlation, but conventionally termed convolution.
Given an input image \(I\) and a kernel (filter) \(K\), the feature map \(S\) is defined as:
\[S(i, j) = (I * K)(i, j) = \sum_{m} \sum_{n} I(i+m, j+n) K(m, n)\]
Where: * \((i, j)\) are the pixel coordinates. * \((m, n)\) are the kernel offsets.
Pooling Layers
Pooling provides invariance to small translations and reduces dimensionality (downsampling).
Max Pooling
Selects the maximum activation in the receptive field: \[y_{i,j,k} = \max_{(p,q) \in \mathcal{R}_{i,j}} x_{p,q,k}\]
Average Pooling
Calculates the arithmetic mean. Generally, Max Pooling performs better for identifying dominant features (edges, textures).
Activation Functions
Linear convolution is insufficient for approximating non-linear functions.
ReLU (Rectified Linear Unit): \[f(x) = \max(0, x)\]
- Sparsity: Activations \(< 0\) are zeroed out.
- Gradient Propagation: Mitigates vanishing gradient problem compared to Sigmoid/Tanh.
Variants: Leaky ReLU, ELU, GELU (Gaussian Error Linear Unit).
Backpropagation in CNNs
Training requires computing gradients w.r.t weights \(W\) using the Chain Rule.
For a convolution layer \(l\): \[\frac{\partial L}{\partial W^{(l)}} = \frac{\partial L}{\partial \text{out}^{(l)}} * \text{in}^{(l)}\]
Where the gradient is computed via convolution between the incoming error signal and the input activations from the previous layer.
Implementation: PyTorch Snippet
import torch
import torch.nn as nn
class SimpleCNN(nn.Module):
def __init__(self):
super(SimpleCNN, self).__init__()
# Feature Extraction
self.features = nn.Sequential(
nn.Conv2d(in_channels=1, out_channels=32, kernel_size=3, padding=1),
nn.ReLU(),
nn.MaxPool2d(kernel_size=2, stride=2),
nn.Conv2d(32, 64, kernel_size=3, padding=1),
nn.ReLU(),
nn.MaxPool2d(2, 2)
)
# Classifier
self.classifier = nn.Sequential(
nn.Flatten(),
nn.Linear(64 * 7 * 7, 128), # Assuming 28x28 input
nn.ReLU(),
nn.Linear(128, 10)
)
def forward(self, x):
x = self.features(x)
x = self.classifier(x)
return x
Recurrent Neuronal Network
¿Por qué RNNs?
Las redes neuronales tradicionales (MLP, CNN) asumen que las entradas y salidas son independientes.
El problema: ¿Cómo procesamos datos donde el orden importa?
- Traducción de idiomas (La gramática depende del contexto previo).
- Audio y Voz (La onda sonora es continua).
- Series de Tiempo (El valor de hoy depende de ayer).
- Genómica (Secuencias de ADN).
La Celda Recurrente (RNN Cell)
La diferencia fundamental es la memoria. La RNN procesa la entrada actual (\(x_t\)) considerando el estado anterior (\(a_{t-1}\)).
![]()
Figura 1
Desenrollando en el Tiempo (Unrolling)
Una RNN es simplemente la misma celda ejecutada múltiples veces. \(a_t\) pasa información de un paso al siguiente.
![]()
Figura 2
Resumen Visual
![]()
Figura 3
Conclusión: Las RNNs unifican el aprendizaje sobre datos secuenciales aprendiendo parámetros (\(W\)) que se comparten a través del tiempo.
Basic Concepts
![]()
Figura 4
Basic Concepts
Let’s make a general structure of the problem
![]()
Figura 5
Basic Concepts
![]()
Figura 6